5420 Anomaly Detection, Fall 2020¶
Assignment 11: SHAP Values using Randon Forest¶
https://courseworks2.columbia.edu/courses/120274/assignments/503952?module_item_id=1036289
Submitted by: Harsh Dhanuka, hd2457
Objectives¶
It is important in any data science project to define the objective as specific as possible. Below let's write it from general to specific. This will direct our analysis.
- Build a model for Loan Default Prediction, to interpret the SHAP values
- Build a model for Loan Default Prediction, to interpret the SHAP values of the variables, using the top few variables as discovered in the previous RF and GBM models.
- Build a model for Loan Default Prediction, to interpret the SHAP values of the variables, by running the Random Forest model from sklearn package.
- Build a model for Loan Default Prediction, to interpret the SHAP values of the variables, and display the following charts: The summary_plot, The dependence_plot, The force_plot for 4 individual observations
Please click Section 4 to directly go to the RF Model and SHAP Values¶
Table of Contents¶
Section 1: Initial Steps
Section 2: Data Cleaning and Preparation, Feature Engineering
- Section 2: Data Cleaning and Preparation, Feature Engineering
- Section 2.1: Convert the DateTime column AP005 to the relevant formats of Year, Month, Day
- Section 2.2: Convert the misrepresented numerical categorical variables back to relevant category/object format
- Section 2.3: Drop all blank value and id columns
- Section 2.4: Convert all the negative or mis-read values such as -99, etc, to 'nan' for imputation
- Section 2.5: Fill NA's using Iterative Imputer
- Section 2.6: Convert the loan amount column CR009 to a category variable and bin appropriately
Section 3: EDA of all variables and binning
- Section 3: EDA and binning
- Section 3.1: Check the Correlation using correlation plot
- Section 3.2: Create categorical variables using qcut, and assign function to bin all continuous variables
- Section 3.3: Distributions of the numerical data, and the % Y by X which is the mean column for all the Nategorical columns here
- Section 3.4: Distributions of the categorical data, and the % Y by X which is the mean column for all the Categorical columns here
- Section 3.5: EDA - Showing the distribution of X
- Section 3.6: EDA - Showing the Distribution of Y by another Categorical Variable X
- Section 3.7: EDA - Showing interaction of 2 or 3 variables
Section 4: RF Model and SHAP Value Plots
- Section 4: Build RF Model and find SHAP Values
Section 5: Future Considerations, LIME Model, Conclusion
SHAP VALUES:¶
SHAP Values (an acronym from SHapley Additive exPlanations) break down a prediction to show the impact of each feature. It is a method to explain individual predictions. SHAP is based on the game theoretically optimal Shapley Values. Where could we use this?
- A model says a bank shouldn't loan someone money, and the bank is legally required to explain the basis for each loan rejection (whish is what we will be doing in this assignment)
- A healthcare provider wants to identify what factors are driving each patient's risk of some disease so they can directly address those risk factors with targeted health interventions
Lundberg and Lee (2016) proposed the SHAP value as a united approach to explaining the output of any machine learning model. Three benefits worth mentioning here.
- The first one is global interpretability — the collective SHAP values can show how much each predictor contributes, either positively or negatively, to the target variable. This is like the variable importance plot but it is able to show the positive or negative relationship for each variable with the target (see the SHAP value plot below).
- The second benefit is local interpretability — each observation gets its own set of SHAP values (see the individual SHAP value plot below). This greatly increases its transparency. We can explain why a case receives its prediction and the contributions of the predictors. Traditional variable importance algorithms only show the results across the entire population but not on each individual case. The local interpretability enables us to pinpoint and contrast the impacts of the factors.
- Third, the SHAP values can be calculated for any tree-based model, while other methods use linear regression or logistic regression models as the surrogate models.
SHAP Plot Variable Explanation
1. Initial Steps ¶
# Import all packages
import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
%matplotlib inline
import scipy
import time
import seaborn as sns
sns.set(style="whitegrid")
import warnings
warnings.filterwarnings("ignore")
from sklearn.impute import SimpleImputer
from sklearn import preprocessing
from sklearn.preprocessing import StandardScaler
from sklearn.model_selection import train_test_split
from sklearn import metrics
from sklearn.metrics import classification_report, confusion_matrix
from sklearn.metrics import roc_curve, auc, roc_auc_score, accuracy_score, confusion_matrix
from sklearn.metrics import average_precision_score
from sklearn.metrics import precision_recall_curve
from sklearn.ensemble import RandomForestRegressor
import plotly
import plotly.express as px
from collections import Counter
1.1. Load Data ¶
# Read the data
df = pd.read_csv('/Users/harshdhanuka/Desktop/Columbia Class Matter/SEM 3/5420 Anomaly Detection/Assignment 2 EDA/XYZloan_default_selected_vars.csv')
df.head(2)
1.2. Basic Summary Check ¶
print("Number of rows and columns in the dataset:")
df.shape
# Check basic statistics
print("Basic statistics of the columns are as follows:")
df.describe()
1.3. Basic EDA and considerations ¶
a. Basic EDA of one variable AP006¶
df['AP006'].hist()
df.AP006.hist()
df['AP006'].value_counts()
b. EDA of the target variable loan_default¶
# Check the target variable column
print("The number of 0's and 1's are:")
print(df['loan_default'].value_counts())
df['loan_default'].hist()
c. Check the column data types, and NA's¶
#df.info()
d. Feature considerations from eyeballing the data types¶
- The first 3 variables are ID columns, cannot be used for predictions.
Unnamed: 0,Unnamed: 0.1andid.
They need to be dropped.
AP005is a Date-Time column, which cannot be used for any predictions in the model. Date-Time columns act as an ID column and all have unique values, which misrepresents the variable while making predictions. The reason is that this field almost becomes a unique identifier for each record. It is as if you employ the ‘id’ field in your decision trees.
I will derive year, month, day, weekday, etc. from this field. In some models, we may use ‘year’ as a variable just to explain any special volatility in the past. But we will never use the raw DateTime field as a predictor.
- The following columns have 0 as their value in all entries, and hence, they need to be removed from any model predictions.
TD025, TD026, TD027, TD028, CR012.
- The following columns have the same value in all entries, and hence, they need to be removed from any model predictions.
TD029, TD044, TD048, TD051, TD054, TD055, TD061, TD062.
- Check for categorical fields from the data variable descriptions. Convert the relevant numeric fields to their respective categorical fields:
AP002Gender,
AP003Education Code,
AP004Loan Term,
AP006OS Type,
AP007Application City Level,
AP008Flag if City not Application City,
AP009 Binary format,
MB007 Mobile Brands/type
.
------------------------------------ SECTION BREAK ------------------------------------¶
.
2. Data Cleaning and Preparation, Feature Engineering ¶
2.1. Convert the DateTime column AP005 to the relevant formats of Year, Month, Day ¶
df['AP005'] = pd.to_datetime(df['AP005'])
# Create 4 new columns
df['Loan_app_day_name'] = df['AP005'].dt.day_name()
df['Loan_app_month'] = df['AP005'].dt.month_name()
df['Loan_app_time'] = df['AP005'].dt.time
df['Loan_app_day'] = df['AP005'].dt.day
# Drop old column
df = df.drop(columns = ['AP005'])
df.head(2)
2.2. Convert the misrepresented numerical categorical variables back to relevant category/object format ¶
df["AP002"] = df["AP002"].astype('object')
df["AP003"] = df["AP003"].astype('object')
df["AP004"] = df["AP004"].astype('object')
df["AP006"] = df["AP006"].astype('object')
df["AP007"] = df["AP007"].astype('object')
df["AP008"] = df["AP008"].astype('object')
df["AP009"] = df["AP009"].astype('object')
df["CR015"] = df["CR015"].astype('object')
df["MB007"] = df["MB007"].astype('object')
df['Loan_app_day_name'] = df['Loan_app_day_name'].astype('object')
df['Loan_app_month'] = df['Loan_app_month'].astype('object')
df['Loan_app_time'] = df['Loan_app_time'].astype('object')
df['Loan_app_day'] = df['Loan_app_day'].astype('object')
2.3. Drop all blank value/id columns ¶
df = df.drop(columns = ['Unnamed: 0', 'Unnamed: 0.1', 'id', 'TD025', 'TD026', 'TD027', 'TD028', 'CR012','TD029', 'TD044', 'TD048', 'TD051', 'TD054', 'TD055', 'TD061', 'TD062'])
df.head(2)
2.4. Convert all the negative or mis-read values such as -99, etc, to 'nan' for imputation ¶
As per all the variable description, all the following columns are either counts, lengths, or days. Hence, the negative values such as -999, -99, -98, -1, etc are all mis-read NA's and need to be converted back to 'nan' format.
features_nan = ['AP001',
'TD001', 'TD002', 'TD005', 'TD006', 'TD009', 'TD010',
'TD013', 'TD014', 'TD015', 'TD022', 'TD023', 'TD024', 'CR004', 'CR005',
'CR017', 'CR018', 'CR019', 'PA022', 'PA023', 'PA028',
'PA029', 'PA030', 'PA031', 'CD008', 'CD018', 'CD071', 'CD072', 'CD088',
'CD100', 'CD101', 'CD106', 'CD107', 'CD108', 'CD113', 'CD114', 'CD115',
'CD117', 'CD118', 'CD120', 'CD121', 'CD123', 'CD130', 'CD131', 'CD132',
'CD133', 'CD135', 'CD136', 'CD137', 'CD152', 'CD153', 'CD160', 'CD162',
'CD164', 'CD166', 'CD167', 'CD169', 'CD170', 'CD172', 'CD173', 'MB005']
# Define a function to convert negatives to nan
def convert_to_nan(var):
df[var][df[var] < 0] = np.nan
for i in features_nan:
convert_to_nan(i)
# Verify that the negatives are gone
print("The minimum now stands at 0 for most of the columns, verifying the mis-represented values are gone.")
df[features_nan].describe()
2.5. Fill the NA's using Iterative Imputer ¶
Multivariate imputer that estimates each feature from all the others. A strategy for imputing missing values by modeling each feature with missing values as a function of other features in a round-robin fashion.
Instead of using the traditional simple mean, median or mode method of imputation, I will be using the smart method of Iterative Imputation from sklearn package.¶
The documentation is here: https://scikit-learn.org/stable/modules/generated/sklearn.impute.IterativeImputer.html
from sklearn.experimental import enable_iterative_imputer # noqa
from sklearn.impute import IterativeImputer
Make a subset of the original df with only the numeric/float64 variables which have the NA's¶
df_2 = df[features_nan]
# Verify
df_2.head(3)
'MEDIAN' imputation through iterative imputer¶
imp = IterativeImputer(missing_values=np.nan, sample_posterior=False,
max_iter=10, tol=0.001,
n_nearest_features=None, initial_strategy='median')
imp.fit(df_2)
imputed_data_median = pd.DataFrame(data=imp.transform(df_2),
columns=['AP001',
'TD001', 'TD002', 'TD005', 'TD006', 'TD009', 'TD010',
'TD013', 'TD014', 'TD015', 'TD022', 'TD023', 'TD024', 'CR004', 'CR005',
'CR017', 'CR018', 'CR019', 'PA022', 'PA023', 'PA028',
'PA029', 'PA030', 'PA031', 'CD008', 'CD018', 'CD071', 'CD072', 'CD088',
'CD100', 'CD101', 'CD106', 'CD107', 'CD108', 'CD113', 'CD114', 'CD115',
'CD117', 'CD118', 'CD120', 'CD121', 'CD123', 'CD130', 'CD131', 'CD132',
'CD133', 'CD135', 'CD136', 'CD137', 'CD152', 'CD153', 'CD160', 'CD162',
'CD164', 'CD166', 'CD167', 'CD169', 'CD170', 'CD172', 'CD173', 'MB005'],
dtype='int')
imputed_data_median.head(3)
2.6. Convert the loan amount column CR009 to a category variable and bin appropriately ¶
df['CR009'] = pd.cut(x=df['CR009'], bins=[-1, 100000, 200000, 300000, 400000, 500000, 600000, 700000, 800000, 900000, 1000000, 1500000])
df = df.astype({'CR009':'object'})
df.CR009.value_counts()
.
------------------------------------ SECTION BREAK ------------------------------------¶
.
3. EDA of variables and binning ¶
corr = df[['loan_default', 'AP001', 'TD001', 'TD002', 'TD005', 'TD006', 'TD009', 'TD010', 'TD013', 'TD014', 'TD015', 'TD022', 'TD023', 'TD024']].corr()
f,ax = plt.subplots(figsize=(18,12))
sns.heatmap(corr, annot=True, cmap='Greens', linewidths=.4, fmt= '.1f',ax=ax)
plt.show()
# Remove 1 feeature from a pair which has over 0.7 ratio
# I will not remove variable here, as I am selecting the important variables based on previous RF and GBM models.
corr_var_drop1 = ['TD005', 'TD022', 'TD006', 'TD009', 'TD013', 'TD023', 'TD010', 'TD014']
# df = df.drop(columns = corr_var_drop1)
Check the correlations for the counts, lengths, and days columns¶
I will be using the other variables as they are all Call detail data.
filter_col = [col for col in df if col.startswith('CD')]
filter_col.append('loan_default')
corr = df[filter_col].corr()
f,ax = plt.subplots(figsize=(21,21))
sns.heatmap(corr, annot=True, cmap='Greens', linewidths=.4, fmt= '.1f',ax=ax)
plt.show()
# Remove 1 feature from a pair which has over 0.7 ratio
# I will not remove variable here, as I am selecting the important variables based on previous RF and GBM models.
corr_var_drop2 = ['CD173', 'CD172', 'CD170', 'CD169', 'CD167', 'CD166', 'CD164', 'CD162',
'CD137', 'CD136', 'CD135', 'CD133', 'CD132', 'CD131', 'CD117', 'CD118',
'CD120', 'CD121', 'CD123', 'CD114', 'CD113', 'CD108', 'CD107', 'CD106',
'CD101', 'CD072']
# df = df.drop(columns = corr_var_drop2)
df_bin = df.copy(deep = True)
df_bin.head(2)
# Write a function and loop through
def binning(var):
df_bin[var + '_bin'] = pd.qcut(df_bin[var],15,duplicates='drop').values.add_categories("NoData")
df_bin[var + '_bin'] = df_bin[var + '_bin'].fillna("NoData").astype(str)
df_bin[var + '_bin'].value_counts(dropna=False)
features = ['AP001', # 'AP002', 'AP003', 'AP004', 'AP006', 'AP007',
# 'AP008', 'AP009',
'TD001', 'TD002', 'TD005', 'TD006', 'TD009', 'TD010',
'TD013', 'TD014', 'TD015', 'TD022', 'TD023', 'TD024', 'CR004', 'CR005',
#'CR009', 'CR015',
'CR017', 'CR018', 'CR019', 'PA022', 'PA023', 'PA028',
'PA029', 'PA030', 'PA031', 'CD008', 'CD018', 'CD071', 'CD072', 'CD088',
'CD100', 'CD101', 'CD106', 'CD107', 'CD108', 'CD113', 'CD114', 'CD115',
'CD117', 'CD118', 'CD120', 'CD121', 'CD123', 'CD130', 'CD131', 'CD132',
'CD133', 'CD135', 'CD136', 'CD137', 'CD152', 'CD153', 'CD160', 'CD162',
'CD164', 'CD166', 'CD167', 'CD169', 'CD170', 'CD172', 'CD173', 'MB005'
# 'MB007', 'Loan_app_day_name', 'Loan_app_month', 'Loan_app_time',
# 'Loan_app_day'
]
# categorical variables are commented out
for i in features:
binning(i)
# View the bins of some variables
print(df_bin['TD001_bin'].value_counts(dropna=False))
print(df_bin['TD022_bin'].value_counts(dropna=False))
3.3. Distributions of the numerical data, and the % Y by X which is the mean column for all the numerical columns here ¶
This will help identify if mean or median is a better imputation for NA's and also help bin better manually.¶
Also, this will help in feature selection moving forward.¶
The 'mean' column represents the '% Y by X'.
def plot_X_and_Y(var):
z = df_bin.groupby(var + '_bin')['loan_default'].agg(['count','mean']).reset_index()
z['count_pcnt'] = z['count']/z['count'].sum()
x = z[var + '_bin']
y_mean = z['mean']
count_pcnt = z['count_pcnt']
ind = np.arange(0, len(x))
width = .5
fig = plt.figure(figsize=(16,4))
plt.subplot(121)
plt.bar(ind, count_pcnt, width, color='r')
#plt.ylabel('X')
plt.title(var + ' Distribution')
plt.xticks(ind,x.tolist(), rotation=45)
plt.subplot(122)
plt.bar(ind, y_mean, width, color='b')
#plt.ylabel('Y by X')
plt.xticks(ind,x.tolist(), rotation=45)
plt.tight_layout()
plt.title('Response mean by ' + var)
plt.show()
#for i in features:
# plot_X_and_Y(i)
3.4. Distributions of the categorical data, and the % Y by X which is the mean column for all the Categorical columns here ¶
This will help identify if mean or median is a better imputation for NA's and also help bin better manually.¶
Also, this will help in feature selection moving forward.¶
The 'mean' column represents the '% Y by X'.
features_2 = ['AP002', 'AP003', 'AP004', 'AP006', 'AP007', 'AP008', 'AP009',
'CR009','CR015', 'MB007', 'Loan_app_day_name', 'Loan_app_month',
'Loan_app_day'
]
def plot_X_and_Y_cat(var):
z = df_bin.groupby(var)['loan_default'].agg(['count','mean']).reset_index()
z['count_pcnt'] = z['count']/z['count'].sum()
x = z[var]
y_mean = z['mean']
count_pcnt = z['count_pcnt']
ind = np.arange(0, len(x))
width = .5
fig = plt.figure(figsize=(16,4))
plt.subplot(121)
plt.bar(ind, count_pcnt, width, color='r')
plt.ylabel('X')
plt.title(var + ' Distribution')
plt.xticks(ind,x.tolist(), rotation=45)
plt.subplot(122)
plt.bar(ind, y_mean, width, color='b')
plt.ylabel('Y by X')
plt.xticks(ind,x.tolist(), rotation=45)
plt.tight_layout()
plt.title('Response mean by ' + var)
plt.show()
for i in features_2:
plot_X_and_Y_cat(i)
Obervations:¶
From the above graphs, the following variables seem to be not important, as they do not have a pattern or a trend, or a curve on the '% Y by x' graph:
Loan_app_day_name
Other EDA¶
3.5. Showing the Distribution of X ¶
df_count = df['AP006'].value_counts()
df_count = pd.DataFrame(df_count).reset_index()
df_count.columns = ['AP006 - OS Type','Count']
print(df_count.head())
fig = px.bar(df_count, x = 'AP006 - OS Type', y = 'Count', color = 'AP006 - OS Type',
width=600, height=400,
title = "Distribution of OS type")
fig.show()
df_count = df['AP002'].value_counts()
df_count = pd.DataFrame(df_count).reset_index()
df_count.columns = ['AP002 - Gender','Count']
print(df_count.head())
fig = px.bar(df_count, x = 'AP002 - Gender', y = 'Count', color = 'AP002 - Gender',
width=600, height=400,
title = "Distribution of Gender")
fig.show()
df_count = df['AP003'].value_counts()
df_count = pd.DataFrame(df_count).reset_index()
df_count.columns = ['AP003 - Education','Count']
print(df_count.head())
fig = px.bar(df_count, x = 'AP003 - Education', y = 'Count', color = 'AP003 - Education',
width=600, height=400,
title = "Distribution of Education")
fig.show()
fig = px.box(df, x="TD001",width=1000, height=500,
title = "Distribution of TD001 - TD_CNT_QUERY_LAST_7Day_P2P")
fig.show()
fig = px.box(df, x="MB005",width=1000, height=500,
title = "Distribution of MB005")
fig.show()
3.6. Showing the Distribution of Y by another Categorical Variable X ¶
fig = px.box(df, x="AP007", y="TD001",width=900, height=400,
color = "AP002",
title = "The Distribution of Level Application City by TD_CNT_QUERY_LAST_7Day_P2P")
fig.show()
3.7. Showing interaction of two or three variables ¶
fig = sns.pairplot(df[['AP002', 'AP003', 'AP004']],
hue= 'AP004')
fig
.
------------------------------------ SECTION BREAK ------------------------------------¶
.
4. Build the Random Forest Model and find SHAP Values ¶
Use the non-binned data, fill NA's with Iterative Imputer Median¶
Using the median imputed values from the Iterative Imputer¶
# Over write the NA value columns, with the previously calculated values
df[features_nan] = imputed_data_median
df.head(2)
df.isnull().sum().sum()
Selecting the X or the independent variables, as per the Variable Importance found previously in the RF and GMB models.¶
# Selecting the top 10 variables based on above two Variable Importance graphs
vars_selected = ['loan_default',
'TD013',
'AP003',
'AP004',
'MB007',
'TD009',
'TD005',
'CR015',
'TD014',
'MB005',
'CD123'
]
Make a final dataset with only the 10 most important variables, & the target variable¶
df1 = df[vars_selected]
df1.head(2)
Dummy Code all categorical variables¶
df1 = pd.get_dummies(df1,
drop_first = True)
# Drop the first level, to avoid multi-collinearity
df1.head(2)
df1.shape
4.1. RF Model using sklearn ¶
Make a list of all of the columns, as per the new dummy set, except the target variable¶
X_var_list = df1.columns.to_list()
X_var_list = X_var_list[1:]
# X_var_list
Define the target(Y) and the X variables¶
Y = df1['loan_default']
X = df1[X_var_list]
Split to train and test¶
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25)
Run the Random Forest Model¶
model = RandomForestRegressor(
max_depth = 6,
random_state = 0,
n_estimators=10)
model.fit(X_train, Y_train)
print(model.feature_importances_[:10])
Visualize the feature importances¶
importances = model.feature_importances_
indices = np.argsort(importances)
features = X_train.columns
var_imp_chart = pd.DataFrame()
var_imp_chart['Indices'] = indices
var_imp_chart['Variable'] = features
var_imp_chart['Importance'] = importances
var_imp_chart = var_imp_chart.sort_values(by = 'Importance', axis = 0, ascending = False)
print("The top 10 most important variables are: " + '\n')
var_imp_chart.head(10)
importances = model.feature_importances_
indices = np.argsort(importances)
features = X_train.columns
f, ax = plt.subplots(figsize=(12,40))
plt.title('Feature Importances')
plt.barh(range(len(indices)), importances[indices], color='g', align='center')
plt.yticks(range(len(indices)), [features[i] for i in indices])
plt.xlabel('Relative Importance')
plt.show()
4.2. SHAP Values and SHAP Visualizations ¶
import shap
shap_values = shap.TreeExplainer(model).shap_values(X_train)
# Determine the correlation in order to plot with different colors
corrlist = np.zeros(len(features))
X_train_np = X_train.to_numpy() # our X_train is a pandas data frame. Convert it to numpy
for i in range(0,len(features)):
tmp = np.corrcoef(shap_values[:,i],X_train_np[:,i])
corrlist[i] = tmp[0][1]
# plot it
#shap_v_abs = np.abs(shap_values) # Get the absolute values of all SHAP values
#k = pd.DataFrame(shap_v_abs.mean()).reset_index()
#k.columns = ['Variables','abs_SHAP']
#k2 = k.merge()
4.2.1. Global Interpretability ¶
SHAP Variable Importance Plot¶
The shap.summary_plot function with plot_type = ”bar” lets us produce the variable importance plot.
A variable importance plot lists the most significant variables in descending order. The top variables contribute more to the model than the bottom ones and thus have high predictive power.
shap.summary_plot(shap_values, X_train, plot_type = "bar")
df.CR015.value_counts()
Observation:¶
The SHAP Value Variable Importance plot need not be the same as the RF Model Variable Importance plot. For our analysis, we will only consider the SHAP Value plot, as it is more valuable and credible.
As we see, the most important variable is TD013 (Count of Queries in Last 6 Months), followed by the level 12 of AP004 (Loan Term), which is the 12 years of Loan term.
4.2.2. SHAP Summary Plot ¶
The SHAP value summary plot can further show the positive and negative relationships of the predictors with the target variable, and their influence on the predictions.
About the Summary Plot -¶
Y axis: The Left side, or the Y axis shows the different variables, sorted as per the variable significance, from top to bottomColor: Red means a high value of that variable, and Blue means a low value of that variableX axis: This is the SHAP value, which measures the contribution of that partocular variable to the target, which is prediction of the loan default.Points/Dots: Each dot represents each (one) observation in the data (X Train)
A positive, or a high SHAP value means that the variable has a strong, positive impact to the target.¶
shap.summary_plot(shap_values, X_train)
Business Insights of Top 5 variables:¶
TD013- This variable, or theCount of Queries in Last 6 months, has a positive relation with the loan default, in a way that, if this value is high (more red), it is likely that the value of loan default will be high. This means that a high count of queries in last 6 months is likely to be a loan default.AP004_12- This Variable, or theLoan Term of 12 years(has 4 levels - 3, 6, 9, and 12 years), has a positive relation to the loan default, in a way that, if the if this value is high (more red), it is likely that the value of loan default will be high. This means that a loan term of 12 years is likely to be a loan default.AP003_3- This variable, or theEducation Code Level 3(has 5 levels - 1, 3, 4, 5, 6), has a negative relation with the loan default, in a way that, if this value is high (more red), it is very less likely that the value of loan default will be high. This means that a higher education code or level 3 education is less likely to be a loan default.AP003_4- This variable, or theEducation Code Level 4(has 5 levels - 1, 3, 4, 5, 6), has a negative relation with the loan default, in a way that, if this value is high (more red), it is very less likely that the value of loan default will be high. This means that a higher education code or level 3 education is less likely to be a loan default.CR015_6- This variable, or theMonths of Credit Card MOB Max Level 6(has 5 levels - 2, 3, 4, 5, 6), has a negative relation with the loan default, in a way that, if this value is high (more red), it is very less likely that the value of loan default will be high. This means that a more months of credit card max or level 6 is less likely to be a loan default.
Before giving out a loan, or rejecting a loan, the above mentioned explanations will be very handy.¶
Similarly, we can interpret the other variables.
Finally, we can describe the model. A high chance of loan default is associated with the following characteristics, as per the above given explanations:¶
- High TD013
- High AP004_12
- Low AP003_3
- Low AP003_4
- Low CR015_6
- Low MB005
- Low CD123
- High TD009
- High TD005
- High AP004_9
- High TD014
- High AP004_6
Show a simple summary plot, with only the positive vs the negative impacts of the variables.¶
The red color means a positive impact, and the blue means a negative impact.
def ABS_SHAP(df_shap,df):
#import matplotlib as plt
# Make a copy of the input data
shap_v = pd.DataFrame(df_shap)
feature_list = df.columns
shap_v.columns = feature_list
df_v = df.copy().reset_index().drop('index',axis=1)
# Determine the correlation in order to plot with different colors
corr_list = list()
for i in feature_list:
b = np.corrcoef(shap_v[i],df_v[i])[1][0]
corr_list.append(b)
corr_df = pd.concat([pd.Series(feature_list),pd.Series(corr_list)],axis=1).fillna(0)
# Make a data frame. Column 1 is the feature, and Column 2 is the correlation coefficient
corr_df.columns = ['Variable','Corr']
corr_df['Sign'] = np.where(corr_df['Corr']>0,'red','blue')
# Plot it
shap_abs = np.abs(shap_v)
k=pd.DataFrame(shap_abs.mean()).reset_index()
k.columns = ['Variable','SHAP_abs']
k2 = k.merge(corr_df,left_on = 'Variable',right_on='Variable',how='inner')
k2 = k2.sort_values(by='SHAP_abs',ascending = True)
colorlist = k2['Sign']
ax = k2.plot.barh(x='Variable',y='SHAP_abs',color = colorlist, figsize=(8,40),legend=False)
ax.set_xlabel("SHAP Value (Red = Positive Impact)")
ABS_SHAP(shap_values,X_train)
4.2.3. SHAP Variable Dependence Plot ¶
To understand how a single feature effects the output of the model we can plot the SHAP value of that feature vs. the value of the feature for all the examples in a dataset. Since SHAP values represent a feature's responsibility for a change in the model output, the plot(s) below represents the change in predicted loan default as the partocular variable changes.
To help reveal the interactions, dependence_plot automatically selects another feature for coloring. The SHAP plot function automatically includes another variable that our chosen variable interacts most with.
We may ask how to show a partial dependence plot. The partial dependence plot shows the marginal effect one or two features have on the predicted outcome of a machine learning model (J. H. Friedman 2001). It tells whether the relationship between the target and a feature is linear, monotonic or more complex.
In the dependence plots -
Left Y axis- The SHAP Value or the impact of the variable on the targetRight Y axis- The variable which highly correlated wiht the main predictor variable. It gives the variable which interacts most with the main variable being studied.X axis- The variable itself, the actual valuesColor- The value of the second variable, high or low. Red is the high value, Blue is low value
Below, I will show the variable dependence plots for 5 important variables:¶
Dependence Plot for TD013 - Count of Queries in Past 6 Months¶
shap.dependence_plot("TD013", shap_values, X_train)
Business Insights:¶
In this case, we see that:
- As
TD013or the count of queries in last 6 months increases, the value impact on loan default increases, making it more likely to be a default. There is a positive relation between the variable and the loan default value. - If the value of
AP003_4or the Education Code Level 4 is low (more blue), it leads to a increase in the value of the loan default, making it likely to be a default. - Looking at the majority of the points, it can be said that
both the variableshave a different impacts on the loan default value, and if both have high and low values respectively, it is very likely that it will be a loan default. There are more blue dots (low value of AP003_4, in the area where TD013 is high.
Although, it is to note that TD013 has a much larger impact on the value of loan default than the AP003_4.
The plot shows there is an approximately linear and positive trend between “TD013” and the target variable loan default, and “TD013” interacts with “AP003_4” frequently.
A high chance of loan default is associated with a high value for TD013 and low for AP003_4, as per the above given explanations. Hence, giving out of loan should be considered.
Final Insight: A high count of queries in past 6 months, is more likely to result in a loan default and low value of education code level 4 years is also more likely to result in a default.¶
Dependence Plot for AP003_4 - Education Code 4 (out of total 5 levels)¶
shap.dependence_plot("AP003_4", shap_values, X_train)
Business Insights:¶
In this case, we see that:
- The variable being evaluated is
AP003_4ir the education code, at level 4, which is a binary variable. Hence, the graphs only shows two values for the variable, 0 or 1. - If the AP003_4 is 1, which means true for education level code 4, the impact on loan default is distributed but lower, making it less likely to be a default. There is a negative relation between the variable and the loan default value.
- If the value of
TD013or count of queries in last 6 months is high (more red), the impact on loan default is distributed, but the value of loan default is generally the highest for a high value of TD013, making it likely to be a default. - Looking at the majority of the points, it can be said that
both the variableshave different impacts on the loan default value, and if AP003_4 has low values coupled with high values of TD013, it is very likely that it will be a loan default.
Although, it is to note that TD013 has a much larger impact on the value of loan default than the AP003_4.
The plot shows there is a binary trend, and "AP003_4" interacts with “TD013” at eitherr 0 or 1.
A high chance of loan default is associated with a high value for TD013 and low value of AP003_4, as per the above given explanations. Hence, giving out of loan should be considered.
Final Insight: A higher education, such as code 4, is less likely to result in a loan default.¶
Dependence Plot for MB005 - Years phone is active¶
shap.dependence_plot("MB005", shap_values, X_train)
Business Insights:¶
In this case, we see that:
- As
MB005or the number of years the phone is active increases, the value impact on loan default decreases, making it less likely to be a default. There is a negative relation between the variable and the loan default value. - If the value of
AP003_3or the education code level 3 is high (more red), it still has no direct relation to the value of the loan default, however, towards the region where the loan default value is high, there are many blue dots, making it less likely to be a default predictor. - Looking at the majority of the points, it can be said that
both the variableshave different impacts on the loan default value, and if AP003_3 has low values coupled with low values of MB005, it is very likely that it will be a loan default.
Although, it is to note that AP003_3 or loan term has a much larger impact on the value of loan default than the MB005 or years of phone active.
The plot shows there is an approximately downward linear trend between “MB005” and the target variable loan default, and “MB005” interacts with “AP003_3” frequently.
A high chance of loan default is associated with a low value for MB005 and a low value for AP003_3, as per the above given explanations. Hence, giving out of loan should be considered.
Final Insight: A low number of years of phone active, is more likely to result in a loan default.¶
Dependence Plot for CD123 - Count of Distinct Outbound Calls in past 3 months¶
shap.dependence_plot("CD123", shap_values, X_train)
Business Insights:¶
In this case, we see that:
- As
CD123or the count of distinct outbound calls in past 3 months increases, the value impact on loan default decreases, making it less likely to be a default. There is a negative relation between the variable and the loan default value. However, after a certain points, the relation stabilizes. - If the value of
TD013or count of queries in past 6 months is high (more red), it leads to a increase in the value of the loan default, making it likely to be a default. - Looking at the majority of the points, it can be said that
both the variableshave a different impact on the loan default value, and if CD123 has low values coupled with high values of TD013, it is very likely that it will be a loan default.
Although, it is to note that TD013 or counnt of queries has a much larger impact on the value of loan default than the CD123.
The plot shows there is a mixed or hybrid trend between “CD123” and the target variable loan default, and “CD123” interacts with “TD013” frequently.
A high chance of loan default is associated with a low value for CD123 and a high value for TD013, as per the above given explanations. Hence, giving out of loan should be considered.
Final Insight: A low count of distinct outbound calls in past 3 months, is more likely to result in a loan default.¶
Dependence Plot for TD009 - Count of queries in the past 3 months¶
shap.dependence_plot("TD009", shap_values, X_train)
Business Insights:¶
In this case, we see that:
- As
TD009or the count of queries in last 3 months increases, the value impact on loan default increases, making it more likely to be a default. There is a positive relation between the variable and the loan default value. - If the value of
TD013or the count of queries in last 6 months is high (more red), it leads to a increase in the value of the loan default, making it likely to be a default. - Looking at the majority of the points, it can be said that
both the variableshave a positive impact on the loan default value, and if both have high values, it is very likely that it will be a loan default. There are more red dots (high value of TD013, in the area where TD009 is high.
Although, it is to note that TD013 has a much larger impact on the value of loan default than the TD009.
The plot shows there is an approximately linear and positive trend between “TD009” and the target variable loan default, and “TD009” interacts with “TD013” frequently.
A high chance of loan default is associated with a high value for TD009 and TD013, as per the above given explanations. Hence, giving out of loan should be considered.
Final Insight: A high count of queries in past 3 months, is more likely to result in a loan default.¶
Dependence Plots for Multiple variables, shown together¶
For the top 6 variables, it shows the other top 6 variables it interacts with, and their impact on the loan default value.
# Here, I am taking only the first 2,000 rows to save run time
shap_interaction_values = shap.TreeExplainer(model).shap_interaction_values(X_train.iloc[:2000,:])
shap.summary_plot(shap_interaction_values, X_train.iloc[:2000,:])
4.2.4. Local Interpretability ¶
Individual SHAP Value Plot (Force Plots)¶
The force plots help explain the predicted value for each observation.
In order to show how the SHAP values can be done on individual cases, I will execute on several random observations. I randomly chose a few observations in as shown below:
X_output = X_test.copy()
X_output.loc[:,'predict'] = np.round(model.predict(X_output),2)
random_picks = np.arange(1,4000,500)
S = X_output.iloc[random_picks]
S
# Initialize the jupyter notebook
shap.initjs()
Calculate Inidivual SHAP Values¶
I write a small function shap_plot(j) to execute the SHAP values for the observations.
In the code below, the shap.force_plot() takes three values:
- the base value (explainerModel.expected_value[0])
- the SHAP values (shap_values_Model[j][0])
- the matrix of feature values (S.iloc[[j]])
The base value or the expected value is the average of the model output over the training data X_train. It is the base value used in the following plot.
def shap_plot(j):
explainerModel = shap.TreeExplainer(model)
shap_values_Model = explainerModel.shap_values(S)
p = shap.force_plot(explainerModel.expected_value, shap_values_Model[j], S.iloc[[j]])
return(p)
Print the mean of the important variables, to explian the force plots¶
means = pd.DataFrame(X_train.mean()).reset_index()
means.columns = ['Variable', 'Mean']
print(means[means['Variable'] == 'TD013'])
print()
print(means[means['Variable'] == 'CR015_6'])
print()
print(means[means['Variable'] == 'MB005'])
print()
print(means[means['Variable'] == 'TD005'])
print()
print(means[means['Variable'] == 'TD009'])
print()
print(means[means['Variable'] == 'AP004_12'])
print()
print(means[means['Variable'] == 'AP003_3'])
print()
print(means[means['Variable'] == 'AP003_4'])
print()
print(means[means['Variable'] == 'CR015_3'])
print()
print(means[means['Variable'] == 'CD123'])
# Mean or the base value of Y
round(Y_test.mean(), 2)
Individual SHAP Plot (Force Plot) 1¶
shap_plot(0)
Business Insights:¶
Output value: This is the prediction for that observation, which here is 0.15, lower than the mean
Base value: The base value E (y_hat) is "the value that would be predicted if we did not know any features for the current output." In other words, it is the mean prediction, or mean (y_hat). So the mean prediction of Y_test is 0.19.
Features: The above plot shows features that contributes to push the final prediction away from the base value.
Red/blue colors: Those features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue.
Features Explanation:¶
Negative (Blue)
TD013: It has a negative impact on the loan default value. The TD013 or the Count of Queries in the past 6 months observation is 2 which is lower than the average value 6.81. So it pushes the prediction to the left, away from the base value or mean value. As we saw in the summary plot, a high value of this variable results in a likely loan default, hence here, since this value is low, it pushes towards a less likely loan default, to the left.CR015_6: It has a negative impact on the loan default value. The CR015_6 or the Months of Credit Card MOB Max Level 6, has a value of 1 here, which is higher than its average of 0.34, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.MB005: It has a negative impact on the loan default value. The MB005 or the Years Phone is active, has a value of 11 here, which is higher than its average of 5.97, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.
Positive (Red)
CD123: It has a positive impact on the loan default value. The CD123 or Count of Distinct Outbound call in past 3 months, has a value of 32 here, which is lower than its average of 121.59, it pushes the predicted loan default value to the right. As we saw in the summary plot, a low value of this variable results in a more likely loan default, hence here, since this value is low, it pushes towards a more likely loan default, to the right.
Individual SHAP Plot (Force Plot) 2¶
shap_plot(1)
Business Insights:¶
Output value: This is the prediction for that observation, which here is 0.23, higher than the mean
Base value: The base value E (y_hat) is "the value that would be predicted if we did not know any features for the current output." In other words, it is the mean prediction, or mean (y_hat). So the mean prediction of Y_test is 0.19.
Features: The above plot shows features that contributes to push the final prediction away from the base value.
Red/blue colors: Those features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue.
Features Explanation:¶
Negative (Blue)
AP003_3: It has a negative impact on the loan default value. The AP003_3 or Education Code Level 3, has a value of 1 here, which is higher than its average of 0.29, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.CD123: It has a negative impact on the loan default value. The CD123 or Count of Distinct Outbound call in past 3 months, has a value of 274 here, which is higher than its average of 121.59, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.
Positive (Red)
TD013: It has a positive impact on the loan default value. The TD013 or the Count of Queries in the past 6 months observation is 10 which is higher than the average value 6.81. So it pushes the prediction to the right, towards the base value or mean value. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.AP004_12: It has a positive impact on the loan default value. The AP004_12 or a Loan term of 12 years, has a value of 1 here, which is higher than its average of 0.87, it pushes the predicted loan default value to the right. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.
Individual SHAP Plot (Force Plot) 3¶
shap_plot(2)
Business Insights:¶
Output value: This is the prediction for that observation, which here is 0.15, lower than the mean
Base value: The base value E (y_hat) is "the value that would be predicted if we did not know any features for the current output." In other words, it is the mean prediction, or mean (y_hat). So the mean prediction of Y_test is 0.19.
Features: The above plot shows features that contributes to push the final prediction away from the base value.
Red/blue colors: Those features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue.
Features Explanation:¶
Negative (Blue)
TD013: It has a negative impact on the loan default value. The TD013 or the Count of Queries in the past 6 months observation is 2 which is lower than the average value 6.81. So it pushes the prediction to the left, away from the base value or mean value. As we saw in the summary plot, a low value of this variable results in a less likely loan default, hence here, since this value is low, it pushes towards a less likely loan default, to the left.CR015_6: It has a negative impact on the loan default value. The CR015_6 or the Months of Credit Card MOB Max Level 6, has a value of 1 here, which is higher than its average of 0.34, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.
Positive (Red)
AP004_12: It has a positive impact on the loan default value. The AP004_12 or a Loan term of 12 years, has a value of 1 here, which is higher than its average of 0.87, it pushes the predicted loan default value to the right. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.CD123: It has a positive impact on the loan default value. The CD123 or Count of Distinct Outbound call in past 3 months, has a value of 61 here, which is lower than its average of 121.59, it pushes the predicted loan default value to the right. As we saw in the summary plot, a low value of this variable results in a more likely loan default, hence here, since this value is low, it pushes towards a more likely loan default, to the right.
Individual SHAP Plot (Force Plot) 4¶
shap_plot(3)
Business Insights:¶
Output value: This is the prediction for that observation, which here is 0.18, near to the mean
Base value: The base value E (y_hat) is "the value that would be predicted if we did not know any features for the current output." In other words, it is the mean prediction, or mean (y_hat). So the mean prediction of Y_test is 0.19.
Features: The above plot shows features that contributes to push the final prediction away from the base value.
Red/blue colors: Those features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue.
Features Explanation:¶
Negative (Blue)
TD013: It has a negative impact on the loan default value. The TD013 or the Count of Queries in the past 6 months observation is 3 which is lower than the average value 6.81. So it pushes the prediction to the left. As we saw in the summary plot, a low value of this variable results in a less likely loan default, hence here, since this value is low, it pushes towards a less likely loan default, to the left.MB005: It has a negative impact on the loan default value. The MB005 or the Years Phone is active, has a value of 6 here, which is higher than its average of 5.97, it pushes the predicted loan default value to the left. As we saw in the summary plot, a high value of this variable results in a less likely loan default, hence here, since this value is high, it pushes towards a less likely loan default, to the left.
Positive (Red)
CD123: It has a positive impact on the loan default value. The CD123 or Count of Distinct Outbound call in past 3 months, has a value of 53 here, which is lower than its average of 121.59, it pushes the predicted loan default value to the right. As we saw in the summary plot, a low value of this variable results in a more likely loan default, hence here, since this value is low, it pushes towards a more likely loan default, to the right.AP004_12: It has a positive impact on the loan default value. The AP004_12 or a Loan term of 12 years, has a value of 1 here, which is higher than its average of 0.87, it pushes the predicted loan default value to the right. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.
Individual SHAP Plot (Force Plot) 5¶
shap_plot(4)
Business Insights:¶
Output value: This is the prediction for that observation, which here is 0.30, higher than the mean
Base value: The base value E (y_hat) is "the value that would be predicted if we did not know any features for the current output." In other words, it is the mean prediction, or mean (y_hat). So the mean prediction of Y_test is 0.19.
Features: The above plot shows features that contributes to push the final prediction away from the base value.
Red/blue colors: Those features that push the prediction higher (to the right) are shown in red, and those pushing the prediction lower are in blue.
Features Explanation:¶
Positive (Red)
TD013: It has a positive impact on the loan default value. The TD013 or the Count of Queries in the past 6 months observation is 7 which is higher than the average value 6.81. So it pushes the prediction to the right. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.AP003_3: It has a positive impact on the loan default value. The AP003_3 or Education Code Level 3, has a value of 0 here, which is lower than its average of 0.29, it pushes the predicted loan default value to the right. As we saw in the summary plot, a low value of this variable results in a more likely loan default, hence here, since this value is low, it pushes towards a more likely loan default, to the right.AP004_12: It has a positive impact on the loan default value. The AP004_12 or a Loan term of 12 years, has a value of 1 here, which is higher than its average of 0.87, it pushes the predicted loan default value to the right. As we saw in the summary plot, a high value of this variable results in a more likely loan default, hence here, since this value is high, it pushes towards a more likely loan default, to the right.AP003_4: It has a positive impact on the loan default value. The AP003_4 or Education Code Level 4, has a value of 0 here, which is lower than its average of 0.17, it pushes the predicted loan default value to the right. As we saw in the summary plot, a low value of this variable results in a more likely loan default, hence here, since this value is low, it pushes towards a more likely loan default, to the right.
.
------------------------------------ SECTION BREAK ------------------------------------¶
.
5. Future Considerations, LIME Model, Conclusion ¶
There are also other plots, which might give a better explanation of the model. However, all models, do not support all graphs ad we need to extra careful in using them.¶
These are just for illustrative purpose:
https://docs.h2o.ai/h2o/latest-stable/h2o-docs/explain.html
Heatmap
Correlation Map
Individual Conditional Expectation (ICE)
LIME Model¶
LIME (Local Interpretable Model-agnostic Explanations) builds sparse linear models around an individual prediction in its local vicinity. This is documented in Lundberg and Lee (2016) that LIME is actually a subset of SHAP but lacks the same properties.
The Advantage of LIME over SHAP — SPEED¶
Readers may ask: “If SHAP is already a united solution, why should we consider LIME?”
The two methods emerge very differently. The advantage of LIME is speed. LIME perturbs data around an individual prediction to build a model, while SHAP has to compute all permutations globally to get local accuracy. Further, the SHAP Python module does not yet have specifically optimized algorithms for all types of algorithms (such as KNNs), as I have documented in “Explain Any Models with the SHAP Values — Use the KernelExplainer” that test models in KNN, SVM, Random Forest, GBM, or the H2O module.
X_var_list = df1.columns.to_list()
X_var_list = X_var_list[1:]
# X_var_list
Y = df1['loan_default']
X = df1[X_var_list]
X_train, X_test, Y_train, Y_test = train_test_split(X, Y, test_size = 0.25)
model = RandomForestRegressor(
max_depth = 6,
random_state = 0,
n_estimators=10)
model.fit(X_train, Y_train)
import lime
import lime.lime_tabular
X_featurenames = X.columns
explainer = lime.lime_tabular.LimeTabularExplainer(np.array(X_train),
feature_names = X_featurenames,
class_names = ['loan_default'],
#categorical_features = ['AP003', 'AP004', 'MB007', 'CR015'],
verbose=True, mode='regression')
Interpret the first record¶
exp = explainer.explain_instance(X_test.iloc[0], model.predict)
exp.as_pyplot_figure()
Business Insight:¶
- Green/Red color: features that have positive correlations with the target are shown in green, otherwise red.
Coefficients of the LIME model¶
pd.DataFrame(exp.as_list())
Show all the results in a notebook-like format¶
exp.show_in_notebook(show_table = True, show_all = False)
Observations:¶
- The LIME model intercept: -0.129,
- The LIME model prediction: “Prediction_local 0.153”
- The original random forest model prediction: “Right: 0.215”.
Conclusion:¶
SHAP Values are a very important methodology to understand and explian the variables which result a certain prediction value in a random forest model.
Three benefits worth mentioning here:
- The first one is global interpretability — the collective SHAP values can show how much each predictor contributes, either positively or negatively, to the target variable. This is like the variable importance plot but it is able to show the positive or negative relationship for each variable with the target (see the SHAP value plot below).
- The second benefit is local interpretability — each observation gets its own set of SHAP values (see the individual SHAP value plot below). This greatly increases its transparency. We can explain why a case receives its prediction and the contributions of the predictors. Traditional variable importance algorithms only show the results across the entire population but not on each individual case. The local interpretability enables us to pinpoint and contrast the impacts of the factors.
- Third, the SHAP values can be calculated for any tree-based model, while other methods use linear regression or logistic regression models as the surrogate models.